1. Field of the Invention
The present invention relates to a device and method for detecting a motion vector of a camcorder (camera recorder). More specifically, the present invention relates to detection of a motion vector for the purpose of correcting an unstable image of a camcorder caused by unstable hands.
2. Description of the Prior Art
Detection of a motion vector from a dynamic image signal is an essential technique in the compression, recognition, stabilization, etc. of an image.
When a portable video camera is used in conjunction with a VCR (videocassette recorder) to take a picture, an image input to the camera is likely to be unstable and shaking, especially when the user is walking or in a moving vehicle. A common problem of camcorders is that when a picture is highly magnified, the instability caused by unstable hands is more pronounced.
A solid state pick-up device is widely used as an input device of a camcorder. The solid state pick-up device is an image pick-up device made from a semiconductor chip which does not use electron beams. There are two types of solid state pick-up device--a MOS type which uses a metal oxide semiconductor (MOS) transistor in a light receiver, and a CCD type (charge coupled device).
In the following, an existing image correction system will be explained in connection with the accompanying drawings, wherein FIG. 1 is a block diagram showing a conventional image correction system, and FIG. 2 is a conceptual diagram showing a conventional method of one-dimensionally detecting a motion vector of an image.
As shown in FIG. 1, a conventional image correction system for the purpose of stabilizing an image includes:
a CCD imager 11 (a solid state pick-up device) which picks up an input image in a solid state device through an optical system, scans the image electronically within the solid state device, and converts the image into an electrical signal to be outputted; PA1 an analog/digital converter 12 (referred to as A/D converter hereinafter) which converts an analog signal outputted from the CCD imager 11 into a digital signal to be outputted; PA1 a camera signal processing unit 13 which converts a signal outputted from the A/D converter into a color and brightness signal to be outputted; PA1 a motion vector detecting unit 14 which, from a signal outputted from the A/D converter 12, detects a motion vector caused by unstable hands to output a motion vector; PA1 a memory control unit 15 which receives a motion vector outputted from the motion vector detecting unit 14 to control the location of an area to be corrected; PA1 a field memory 16 which holds a field unit (or a frame unit) of image data outputted from the camera signal processing unit 13 and, according to the image correction control by the memory control unit 15, outputs a stabilized image signal; and PA1 a digital/analog converter 17 (referred to as D/A converter hereinafter) which converts a digital image signal outputted from the field memory 16 into an analog image signal to be outputted. PA1 a vertical image pick-up unit which horizontally condenses an inputted image and converts the horizontally condensed image into an electrical signal to be outputted; PA1 a vertical A/D converter which converts an analog signal outputted from the vertical image pick-up unit into a digital signal to be outputted; PA1 a vertical delay register which holds a signal outputted from the vertical A/D converter for a given times then outputs the signal; PA1 a vertical line memory which holds a signal outputted from the vertical delay register; PA1 a vertical correlation arithmetic unit which receives a signal outputted from the vertical A/D converter and a signal from the vertical line memory and calculates correlation values between the two signals to output a vertical motion vector; PA1 a horizontal image pick-up unit which vertically condenses an inputted image and converts the vertically condensed image into an electrical signal to be outputted; PA1 a horizontal A/D converter which converts an analog signal outputted from the horizontal image pick-up unit into a digital signal to be outputted; PA1 a horizontal delay register which holds a signal outputted from the horizontal A/D converter for a given time, then outputs the signal; PA1 a horizontal line memory which holds a signal outputted from the horizontal delay register; and PA1 a horizontal correlation arithmetic unit which receives a signal outputted from the horizontal A/D converter and a signal from the horizontal line memory and calculates correlation values between the two signals to output a horizontal motion vector. PA1 initializing correlation values; PA1 picking up an image in a line CCD of the image pick-up unit; PA1 performing an A/D conversion of an xth pixel of an image signal picked up in the line CCD; PA1 defining the limits of a searching area within a line memory; PA1 calculating a correlation value between the xth pixel and a value within the searching area and accumulating the correlation value; PA1 deciding whether the calculation has been completed up to the limits of the searching area; PA1 increasing a searching location parameter in order to return to the step of calculating, when the calculation has not been completed up to the limits of the searching area; PA1 renewing data in the line memory, when the calculation has been completed up to the limits of the searching area; PA1 deciding whether the calculation has been completed on every pixel picked up in the line CCD; PA1 increasing a pixel location parameter in order to return to the step of the A/D conversion of the next pixel, when the calculation has not been completed on every pixel picked up in the line CCD; PA1 detecting the minimum correlation value among the accumulated correlation values, when the calculation has been completed on every pixel picked up in the line CCD; and PA1 outputting the displacement of the minimum correlation value as a motion vector having a particular direction.
In such an image correction system, correlation values are calculated between image data selected out of an image which is continuous in time, and a motion vector is determined from the point where the minimum correlation value results.
In general, a block matching algorithm is widely used to calculate the correlation value. In order to solve the problems of calculative complexity and real-time processing, it has been suggested to use various techniques such as pyramid search, logarithmic search and so on. A two-dimensional block matching algorithm, however, entails a rapid increase in the number of arithmetic operations with an increase in the number of pixels to be considered. On the other hand, when the number of pixels to be considered is decreased, the algorithm is likely to produce erroneous results such as a local minimum in which only a local area is stabilized.
As shown in FIG. 2, a general method has also been used which one-dimensionally extracts and corrects motion data throughout the entire area of the image by projecting an image pattern.
After an image pattern has been projected or accumulated in the direction of horizontal and vertical axes, the accumulation result is compared with the accumulation result of the preceding image pattern to calculate correlation values between them. A displacement quantity between the two fields is determined from the point where the minimum correlation value results. The x and y axes form the basis for an x and y coordinate system which represents horizontal and vertical axes, respectively, of a field image. Assuming that M is the number of horizontal pixels of an image, and that N is the number of lines of an image, the horizontal correlation value, for examples can be calculated by an equation such as the following: ##EQU1##
In the equation (1) P.sub.h ' and P.sub.h are line memories for detecting a motion vector, into which the preceding and the present image data are accumulated, respectively. u u.vertline.-S.gtoreq.u.gtoreq.S! is an integer within a searching area .+-.S, and it represents a displacement variable. As shown in the equation (1), in each definition area, if a variable u results in the minimum value among the sums of the differences between the accumulation values of the preceding field and of the present field, the variable u is considered to be the optimum displacement of an image. Compared with a two-dimensional matching algorithm, a one-dimensional signal matching algorithm enables the calculation of a motion vector using fewer arithmetic operations than the two-dimensional algorithm, even when there is a large displacement of an image.
The one-dimensional signal matching algorithm, however, has a time restriction in that the calculation of correlation should be finished before the raster scanning of the next field begins, since the projection or accumulation of the present field image data is completed when the raster scanning of an input image comes to an end point. Further, an inputted pixel should be converted into a low quantized level (a binary signal, etc.) in order to economize on the projection memory. Also, the determination of a threshold value in the conversion of pixels into a binary signal, a processing method through the extraction of contours of an image and so on, can lead to the loss of some pixel data. In addition, two pairs of line memories are required, of which one pair corresponds to N, the number of lines of an input image, and the other one to M, the number of horizontal pixels, and, accordingly, a complex calculating algorithm is required.